Information processing apparatus, information processing method, and storage medium

ABSTRACT

Provided is an information processing apparatus comprising: a first obtaining unit configured to obtain viewpoint information indicating change of a virtual viewpoint corresponding to a virtual viewpoint video generated on the basis of a plurality of images captured from a plurality of directions with a plurality of image capturing apparatuses; a second obtaining unit configured to obtain condition information indicating a condition associated with the amount of change of a background in the virtual viewpoint video; a determination unit configured to determine whether the viewpoint information obtained by the first obtaining unit satisfies the condition indicated by the condition information obtained by the second obtaining unit; and an output unit configured to output information according to the result of the determination by the determination unit.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a technique for evaluatingvisually-induced motion sickness due to a virtual viewpoint video.

Description of the Related Art

There is a virtual viewpoint video technique in which videos capturedwith a plurality of cameras are used to reproduce a video through anon-existent camera virtually placed in a three-dimensional space(hereinafter, referred to as the virtual camera). In the generation of avirtual viewpoint video, the user can set the virtual camera path(movement of the position of the virtual camera along the time axis) toany path. However, the virtual camera path may cause visually-inducedmotion sickness depending on how the virtual camera path is set.

From Japanese Patent Laid-Open No. 2013-21455 (hereinafter, referred toas Document 1), a technique has been known in which the degree ofdiscomfort due to video screen movement is estimated to evaluatevisually-induced motion sickness. In the technique described in Document1, a motion vector is detected from each of a plurality of areas dividedfrom the screen to calculate the screen movement, and the degree ofdiscomfort due to the screen movement is estimated.

However, in the technique described in Document 1, the object(foreground) and the background in the video are not separated from eachother. For this reason, it is impossible to determine whether screenmovement is the result of movement of the object or movement of thebackground. This lowers the accuracy of the visually-induced motionsickness evaluation.

SUMMARY OF THE INVENTION

An information processing apparatus according to an aspect of thepresent invention comprises: a first obtaining unit configured to obtainviewpoint information indicating change of a virtual viewpointcorresponding to a virtual viewpoint video generated on a basis of aplurality of images captured from a plurality of directions with aplurality of image capturing apparatuses; a second obtaining unitconfigured to obtain condition information indicating a conditionassociated with an amount of change of a background in the virtualviewpoint video; a determination unit configured to determine whetherthe viewpoint information obtained by the first obtaining unit satisfiesthe condition indicated by the condition information obtained by thesecond obtaining unit; and an output unit configured to outputinformation according to a result of the determination by thedetermination unit.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of the configuration of animage processing apparatus;

FIG. 2 is a diagram illustrating an example of camera arrangement;

FIG. 3 is a diagram illustrating an example of the logical configurationof the image processing apparatus;

FIG. 4 is a flowchart describing the flow of a process by the imageprocessing apparatus;

FIG. 5 is a flowchart describing the flow of a process by the imageprocessing apparatus;

FIG. 6 is a flowchart describing the flow of a process by the imageprocessing apparatus;

FIGS. 7A to 7D are diagrams illustrating relations between an object andvirtual camera paths;

FIG. 8 is a diagram listing evaluation conditions for evaluatingvisually-induced motion sickness;

FIG. 9 is a diagram illustrating an example of a GUI screen provided bythe image processing apparatus;

FIG. 10 is a diagram illustrating an example of the logicalconfiguration of an image processing apparatus;

FIG. 11 is a flowchart describing the flow of a process by the imageprocess apparatus;

FIGS. 12A and 12B are diagrams illustrating an example of UI screensindicating virtual camera coordinates;

FIGS. 13A and 13B are diagrams illustrating an example of setting alimit on the area within which to change the virtual camera coordinates;and

FIGS. 14A and 14B are diagrams illustrating examples of UI screensindicating possible correction ranges.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described below withreference to the drawings. Note that the following embodiments do notlimit the present invention, and not all the combinations of thefeatures described in these embodiments are necessarily essential for asolution provided by the present invention. Meanwhile, the descriptionwill be given with the same reference sign given to identicalcomponents.

Embodiment 1 <Configuration of Image Processing Apparatus>

FIG. 1 is a diagram illustrating an example of the configuration of animage processing apparatus in embodiment 1. An image processingapparatus 100 illustrated in FIG. 1 includes a CPU 101, a main memory102, a storage unit 103, a display unit 104, and an external I/F unit105, which are connected to each other through a bus 108. The CPU 101 isan arithmetic processing device that controls the whole image processingapparatus 100, and performs various processes by executing variousprograms stored in the storage unit 103 or the like. The main memory 102temporarily stores pieces of data, parameters, and so on to be used inthe various processes and also provides a work area to the CPU 101. Thestorage unit 103 is a large-capacity storage apparatus storing variousprograms and various pieces of data necessary for displaying graphicaluser interfaces (GUIs). As the storage unit 103, a non-volatile memorysuch as a hard disk drive or a silicon disk is used, for example. Thedisplay unit 104 is formed of a liquid crystal panel or the like, anddisplays GUIs for setting a virtual camera path during the generation ofa virtual viewpoint video and so on.

The external I/F unit 105 allows an input unit 106 such as a keyboard, amouse, an electronic pen, a touchscreen, etc. and image capturing units107 such as cameras to be connected to the bus 108, and transmits andreceives video data and control signal data.

Note that the image processing apparatus 100 illustrated in FIG. 1 is amere example, and the present invention is not limited to this. Forexample, the image processing apparatus 100 may not include the displayunit 104 and the image capturing unit 107. In one example, the imageprocessing apparatus 100 may include a display control unit (notillustrated), with which the image processing apparatus 100 may controldisplay by a display apparatus (not illustrated) equivalent to thedisplay unit 104. Also, the image processing apparatus 100 may receiveand transmit video data and control signal data from and to imagecapture apparatuses (not illustrated) equivalent to the image capturingunits 107.

FIG. 2 is a diagram illustrating an example of the image capturing units107 in this embodiment. FIG. 2 illustrates an example in which eightimage capturing units 107 are placed in a sports venue for playingsoccer and so on. Camera viewpoints A to H are directed toward a gazingpoint located at the center of a circle. The image capturing units 107are arranged such that the angle formed between the camera viewpoints ofeach pair of adjacent image capturing units is equal.

FIG. 3 is a diagram illustrating an example of the logical configurationof the image processing apparatus 100. The image processing apparatus100 includes a virtual viewpoint video generation unit 300, a virtualcamera parameter storage unit 310, a virtual viewpoint video storageunit 320, a visually-induced motion sickness evaluation unit 330, and awarning video setting unit 340.

The virtual viewpoint video generation unit 300 generates virtualviewpoint video data following a virtual camera path (a path along whichthe virtual viewpoint is caused to move) designated by the user by usinga plurality of videos (frame images) corresponding to a plurality ofviewpoints captured by image capturing units 107. The generated virtualviewpoint video data is stored in the virtual viewpoint video storageunit 320. Also, virtual camera parameters representing the virtualcamera path used in the generation of the virtual viewpoint video datais stored in the virtual camera parameter storage unit 310. The virtualcamera parameters stored in the virtual camera parameter storage unit310 indicate the positions and directions of the virtual viewpoint at aplurality of time points (frames) included in the playback period of thevirtual viewpoint video. In short, the virtual camera parameters storedin the virtual camera parameter storage unit 310 are viewpointinformation indicating change of the virtual viewpoint during thatplayback period. In the following, for simple description, virtualviewpoint video data to be generated and stored will be simply referredto as a virtual viewpoint video.

The visually-induced motion sickness evaluation unit 330 uses thevirtual viewpoint video stored in the virtual viewpoint video storageunit 320 and the virtual camera path used in the generation of thatvirtual viewpoint video to evaluate visually-induced motion sickness dueto the virtual viewpoint video. The visually-induced motion sicknessevaluation unit 330 identifies a video scene on the basis of the virtualcamera path and the virtual viewpoint video, and then evaluates thevisually-induced motion sickness by using an evaluation conditioncorresponding to the specified video image.

There are videos that are similar to each other but one is likely tocause visually-induced motion sickness while the other is unlikely tocause visually-induced motion sickness. When it comes to a virtualviewpoint video, the relation between the object and the pattern ofmovement of the virtual camera may change whether the virtual viewpointvideo is likely to cause motion sickness. For example, a virtualviewpoint video generated as a scene in which the object is still andthe virtual camera moves around it and a virtual viewpoint videogenerated as a scene in which the camera follows and moves in parallelwith the object are similar. However, the former is likely to causevisually-induced motion sickness while the latter is unlikely to causevisually-induced motion sickness. The visually-induced motion sicknessevaluation unit 330 identifies the video scene and performs avisually-induced motion sickness evaluation by using an evaluationcondition corresponding to the identified video scene. In this way, anaccurate visually-induced motion sickness evaluation can be performed.Specifically, the visually-induced motion sickness evaluation unit 330performs a process using such an evaluation condition that a scene thatis likely to cause visually-induced motion sickness is likely to beevaluated as a video that is likely to cause motion sickness. Detailswill be described later.

The warning video setting unit 340 sets a warning video to be displayedon the display unit 104 depending on the result of the visually-inducedmotion sickness evaluation by the visually-induced motion sicknessevaluation unit 330. The set warning video is displayed on the displayunit 104.

The virtual viewpoint video generation unit 300 includes an input videostorage unit 301, a foreground background separation process unit 302, amask data storage unit 303, an object shape estimation unit 304, and avirtual viewpoint video generation process unit 305. Thevisually-induced motion sickness evaluation unit 330 includes aforeground background analysis unit 331, a video analysis unit 332, avideo evaluation unit 333, a viewing condition storage unit 334, and avideo evaluation condition storage unit 335.

In the logical configuration in FIG. 3, the storage units, which storevarious pieces of data and videos, are implemented in the main memory102 or the storage unit 103, for example. As for the other units of thelogical configuration, the CPU 101 executes programs stored in thestorage unit 103 or the like to function as those units in FIG. 3. Notethat in the example of FIG. 3, the image processing apparatus 100 may beconfigured as an apparatus that implements the visually-induced motionsickness evaluation unit 330, and the other units in the configurationmay be implemented with other apparatuses, for example. In short, thelogical configuration illustrated in FIG. 3 may be a configurationimplemented with a plurality of image processing apparatuses such thatthe processing is distributed.

<Virtual Viewpoint Video Generation Process>

FIG. 4 is a diagram illustrating an exemplary flowchart of a process ofgenerating a virtual viewpoint video performed by the virtual viewpointvideo generation unit 300. The process illustrated in FIG. 4 isimplemented as the CPU 101 reads a predetermined program out of thestorage unit 103, decompresses the read program onto the main memory102, and executes this decompressed program. Alternatively, thefunctions of some or all of the steps in FIG. 4 may be implemented withhardware such as an ASIC or an electronic circuit. Meanwhile, the symbol“S” in the description of each process means a step in the flowchart(the same applies throughout this specification).

In S401, the image capturing units 107 corresponding to the plurality ofviewpoints in FIG. 2 (camera viewpoints A to H) capture images from therespective viewpoints in synchronization with each other and store thepieces of video data from the camera viewpoints (pieces of frame imagedata) in the input video storage unit 301.

In S402, the foreground background separation process unit 302 executesa foreground background separation process on each of the frame imagesfrom the camera viewpoints by extracting, from the frame image, a pixelarea corresponding to a predetermined object as the foreground and theremaining pixel area as the background. Further, the foregroundbackground separation process unit 302 generates mask data formed of thepixel area extracted as the foreground. The mask data is an image inwhich the background is masked and the foreground is extracted.

There are known foreground background separation techniques such as aprocess involving comparing frame images with each other and extractingan object, which is a foreground, on the basis of the amount ofdisplacement of each pixel. For this step, any method may be employed aslong as it is a process that separates the foreground and the backgroundfrom the image.

In S403, the foreground background separation process unit 302 storesthe mask data generated in S402 in the mask data storage unit 303.

In S404, the object shape estimation unit 304 executes a process ofestimating the three-dimensional shape of the object by using the frameimage from each camera viewpoint, parameters indicating the position,orientation, and so on of each image capturing unit 107, and the maskdata. As the estimation method, a publicly known method may be used suchas a visual hull method, which uses information on an object'ssilhouette, or a multi-view stereo method, which uses triangulation. Theobject shape estimation unit 304 also estimates the coordinate positionsof the object within the virtual space for generating the virtualviewpoint video.

In S405, the virtual viewpoint video generation unit 300 sets thevirtual camera parameters in the virtual viewpoint video and stores themin the virtual camera parameter storage unit 310. The virtual cameraparameters include the camera coordinate values, the camera orientation,the focal length, and so on in each of the video frames (frame images)along the time axis inside the virtual space for generating the virtualviewpoint video. Specifically, with the virtual camera parameters, it ispossible to identify at least one of the changes in speed, acceleration,deceleration, direction, position, and orientation of the virtualcamera. The virtual camera parameters are input by the user through theinput unit 106, for example.

In S406, the virtual viewpoint video generation process unit 305generates a virtual viewpoint video in accordance with the set virtualcamera parameters. The virtual viewpoint video is generated by applyinga computer graphics technique to a video in which the estimated objectshape at its estimated position is viewed from the set virtual camera.

In S407, the virtual viewpoint video generation process unit 305 storesthe generated virtual viewpoint video in the virtual viewpoint videostorage unit 320. The virtual viewpoint video generation unit 300generates a virtual viewpoint video in this manner. Note that thevirtual viewpoint video generation process may be performed by anapparatus other than the image processing apparatus 100, which performsa visually-induced motion sickness evaluation. The above is the basicprocedure to the point where a virtual viewpoint video is generated.

<Overview of Visually-Induced Motion Sickness Evaluation Process>

FIG. 5 is a diagram illustrating an exemplary flowchart of avisually-induced motion sickness evaluation process performed by thevisually-induced motion sickness evaluation unit 330. FIG. 5 is aflowchart describing the entire procedure to a point where thevisually-induced motion sickness evaluation unit 330 evaluatesvisually-induced motion sickness due to an obtained virtual viewpointvideo and displays a warning for any frame region that is highly likelyto cause visually-induced motion sickness.

In S501, the visually-induced motion sickness evaluation unit 330obtains a virtual viewpoint video stored in the virtual viewpoint videostorage unit 320. For example, a plurality of virtual viewpoint videoscan be stored in the virtual viewpoint video storage unit 320. Thevisually-induced motion sickness evaluation unit 330 obtains a virtualviewpoint video stored as an evaluation target video from among thevirtual viewpoint videos stored in the virtual viewpoint video storageunit 320.

In S502, the visually-induced motion sickness evaluation unit 330obtains the virtual camera parameters used for the evaluation targetvirtual viewpoint video, which are stored in the virtual cameraparameter storage unit 310.

In S503, the visually-induced motion sickness evaluation unit 330 sets aframe image to be evaluated (hereinafter, also referred to as theevaluation target frame) in the virtual viewpoint video obtained inS501. For example, the visually-induced motion sickness evaluation unit330 may set an evaluation target frame on the basis of a designationfrom the user or set an evaluation target frame at preset timeintervals. The visually-induced motion sickness evaluation unit 330 mayset an evaluation target frame in a case where any of predeterminedscenes is present.

In S504, the visually-induced motion sickness evaluation unit 330 refersto a visually-induced motion sickness evaluation condition for theevaluation target frame set in S503. Then, the visually-induced motionsickness evaluation unit 330 determines the likelihood of occurrence ofvisually-induced motion sickness on the basis of the visually-inducedmotion sickness evaluation condition. Details will be described later.

In S505, the warning video setting unit 340 sets a warning flag for theevaluation target frame set in S503 if the visually-induced motionsickness evaluation unit 330 determines that the evaluation target frameis a frame image that is highly likely to cause visually-induced motionsickness. Namely, the warning video setting unit 340 sets a warningframe for the evaluation target frame set in S503.

In S506, from the frame image determined to be highly likely to causevisually-induced motion sickness, the warning video setting unit 340extracts an image area that can be a cause of the visually-inducedmotion sickness, and sets the image area as a warning display area.

In S507, the visually-induced motion sickness evaluation unit 330determines whether all frame images in the evaluation target virtualviewpoint video have been evaluated. If there is any frame image yet tobe evaluated, the visually-induced motion sickness evaluation unit 330updates the evaluation target frame in S509, and the processes in andafter S504 are repeated.

If the evaluation on all frame images has been completed, then in S508,the visually-induced motion sickness evaluation unit 330 displays theresult of the visually-induced motion sickness evaluation on the virtualviewpoint video on the display unit 104, and the process in thisflowchart is terminated. In S508, a warning is displayed on any frameevaluated to cause visually-induced motion sickness. Note, however, thatthe result of the visually-induced motion sickness evaluation on eachsingle frame may be displayed thereon in the form of a numerical valueor another form of information. Also, for frames evaluated not to causevisually-induced motion sickness, information indicating that evaluationresult may be displayed thereon.

The process described in FIG. 5 represents an example in which avisually-induced motion sickness evaluation is performed on a virtualviewpoint video generated by the virtual viewpoint video generation unit300 in the image processing apparatus 100. Note, however, that thepresent invention is not limited to this. The image processing apparatus100 may receive a virtual viewpoint video generated by another apparatusand evaluate visually-induced motion sickness due to that virtualviewpoint video. Alternatively, the image processing apparatus 100 mayread out a virtual viewpoint video stored in a storage medium notillustrated and evaluate visually-induced motion sickness due to thatvirtual viewpoint video.

<Details of Visually-Induced Motion Sickness Evaluation>

FIG. 6 is a flowchart describing details of the visually-induced motionsickness evaluation process listed in S504 in FIG. 5. Details of thevisually-induced motion sickness evaluation will be described below withreference to the configuration diagram of FIG. 3 and the flowchart ofFIG. 6.

In S601, the foreground background analysis unit 331 performs ananalysis process on the foreground and background image areas in theevaluation target frame (virtual viewpoint video) and separates theevaluation target frame into the foreground and the background. Forexample, the foreground background analysis unit 331 may separate theevaluation target frame into the foreground and the background on thebasis of the pixel-by-pixel differences from preceding and subsequentframe images. Alternatively, the foreground background analysis unit 331may use the mask data of the foreground stored in the mask data storageunit 303 to generate a virtual viewpoint video of the foreground maskdata via a process equivalent to the process performed by the virtualviewpoint video generation unit 300, and use this mask data to separatethe virtual viewpoint video into foreground and background image areas.Still alternatively, in the generation of the virtual viewpoint video bythe virtual viewpoint video generation unit 300, its foreground area maybe converted into mask data and stored, and the foreground backgroundanalysis unit 331 may use this stored mask data to separate theevaluation target frame into foreground and background image areas.

The foreground background analysis unit 331 detects the amounts ofdisplacement of the separated foreground and background image areasbetween frames. For example, the foreground background analysis unit 331detects the amounts of displacement of the foreground and backgroundimage areas between frames by using a commonly known motion vectordetection process. These amounts of displacement are the amounts ofmotion in the video size (pixel) (the amounts of motion in terms ofpixels). Further, the foreground background analysis unit 331 alsoobtains information on the position of the separated foreground in theframe image.

Meanwhile, the process in S601 may be executed for all evaluation targetframes when the visually-induced motion sickness evaluation unit 330obtains the evaluation target virtual viewpoint video. In this case, theprocess in S601 may be changed to a process of obtaining the data thathas already been analyzed and stored. In short, the foregroundbackground separation process may not be performed individually on eachevaluation target frame, but the separation process may be performedcollectively on the evaluation target virtual viewpoint video.Alternatively, only the separation process may be performed in advance,and S601 may simply be the process of detecting the amounts ofdisplacement of the foreground and background image areas betweenframes.

In S602, the video analysis unit 332 performs a virtual camera parameteranalysis process. The video analysis unit 332 obtains the virtual cameraparameters used for the evaluation target frame image in the virtualviewpoint video, and obtains the amount of change in virtual cameracoordinates and the amount of change in camera orientation.

In S603, the video analysis unit 332 obtains a viewing condition storedin the viewing condition storage unit 334. The viewing condition is thecondition under which the virtual viewpoint video is to be viewed by theuser. The viewing condition is set by the user. The viewing conditionincludes at least a display condition and an environmental condition.The display condition is, for example, the size in which the virtualviewpoint video is to be displayed. The environmental condition is, forexample, information on the distance between the user who is to view thevirtual viewpoint video and the displayed video. Other viewingconditions may be included such as the display device's type,brightness, and dynamic range, and the conditions of surroundings in theviewing environment (such as lighting). The viewing condition may alsoinclude profile information on user characteristics such as the age andsex of the user who is to view the virtual viewpoint video and theuser's susceptibility to motion sickness, and so on. Visually-inducedmotion sickness can be dependent on the actual viewing environment inwhich the user views the virtual viewpoint video. For example, if thesame video is viewed on a large screen in a theater or the like and on atablet terminal, a home television set, or the like, motion sickness maybe likely to occur in the former case while motion sickness may beunlikely to occur in the latter case. For this reason, thevisually-induced motion sickness evaluation is performed with theviewing condition taken into consideration.

In S604, the video analysis unit 332 analyzes the video scene of theevaluation target frame. The video analysis unit 332 determines theforeground to be defined as the main object in the evaluation targetframe. Then, the video analysis unit 332 obtains the amounts of changeof the foreground defined as the main object and the virtual camerapath.

In S605, the video analysis unit 332 determines the amounts of apparentmotion of the foreground and the background in the video to be viewed bythe viewer by using the viewing condition obtained in S603. For example,the video analysis unit 332 determines the amounts of apparent motion inthe video by referring to the amounts of displacement of the foregroundand the background obtained by the analysis process in S601, the videodisplay size in which the virtual viewpoint video is to be viewed by theuser, and the distance between the user and the displayed video. Motiondirection vectors and angular speeds in the viewer's view are determinedas the amounts of motion. Note that the background area may be divided,and the amount of apparent motion in each divided area may bedetermined. A speed value other than angular speed may be used instead.

In S606, the video evaluation unit 333 obtains a visually-induced motionsickness evaluation condition that matches the result of the analysis inS604 and the amounts of apparent motion obtained in S605 from amongvisually-induced motion sickness evaluation conditions stored in thevideo evaluation condition storage unit 335. Thus, in this embodiment,the evaluation condition (evaluation parameters) for evaluatingvisually-induced motion sickness is varied in accordance with the videoscene. This will be described below with reference to specific examples.

FIGS. 7A to 7D are diagrams explaining video scenes. The relationsbetween a main object, which is a foreground, and virtual camera pathsin virtual viewpoint videos will be described with reference to FIGS. 7Ato 7D. FIG. 7A is a diagram schematically illustrating a motion vectorin a given frame image 700 in a virtual viewpoint video. In the videoscene of FIG. 7A, an object 701 appears to be still in the center of thevirtual viewpoint video. The background, on the other hand, appears tobe moving from right to left.

FIGS. 7B to 7D are diagrams each illustrating the relation betweenchanges in the position of the object 701 in the three-dimensionalvirtual space in a virtual viewpoint video and changes in the positionand orientation of a virtual camera 703.

FIG. 7B represents a virtual camera path along which the virtual camera703 follows the object 701 that is moving, while maintaining thedistance therebetween within a certain range without changing theorientation.

FIG. 7C represents a virtual camera path along which the virtual camera703 does not greatly change its position with respect to the object 701that is moving, but changes its orientation to follow the motion of theobject.

FIG. 7D represents a virtual camera path along which the virtual camera703 moves around the object 701 that is hardly changing in position,while maintaining the distance therebetween within a certain range.

The relations between the object 701 and the virtual camera paths inFIGS. 7B to 7D are different from each other. However, in each of thevirtual viewpoint videos generated with these camera paths, the motionsof the object and the background are similar to those illustrated FIG.7A. In other words, even in a case where video scenes containsubstantially the same apparent motions as those in the frame image 700,the motions of the object and the background and the virtual camera pathmay be different. Moreover, the degree of visually-induced motionsickness which video viewers experience (likelihood of visually-inducedmotion sickness) from the apparent motions in a video varies.Specifically, as for the degree of visually-induced motion sicknesswhich video viewers experience from the apparent motions in the virtualviewpoint video illustrated in FIG. 7A, is highest with FIG. 7D, so thatFIG. 7D is most likely to cause motion sickness. FIG. 7C and FIG. 7Bcause lower degrees of visually-induced motion sickness in this orderand are therefore videos less likely to cause motion sickness in thisorder.

As described above, even if videos contain the same amounts of motion ofthe object and the background, the degree of visually-induced motionsickness caused by those amounts of motion may vary depending on therelation between the object and the virtual camera path. Thus, theevaluation condition for evaluating whether the video causesvisually-induced motion sickness is preferably changed in accordancewith the video scene. Specifically, the evaluation condition forevaluating visually-induced motion sickness is preferably changed inaccordance with the motions of the object and the background and thevirtual camera path. The motions of the object and the background caninclude at least one of changes in speed, acceleration, deceleration,direction, and position of the object and the background.

In this embodiment, the evaluation condition is determined on the basisof the video scene of the evaluation target frame. Specifically, theevaluation condition for evaluating visually-induced motion sickness isdetermined in accordance with the motions of the object and thebackground in the evaluation target frame and the virtual camera path.Then, a visually-induced motion sickness evaluation is performed on thebasis of the determined evaluation condition.

FIG. 8 is a diagram explaining evaluation conditions for evaluatingvisually-induced motion sickness. In FIG. 8, evaluation conditions withdifferent degrees of visually-induced motion sickness corresponding toamounts of displacement of the foreground defined as the main object andtypes of virtual camera movement track are defined. Evaluationconditions 800 are stored in the video evaluation condition storage unit335 in advance.

The video analysis unit 332 analyzes the video scene and determines theforeground to be defined as the main object from among the plurality offoregrounds present in the evaluation target frame by referring to acondition 803. In this embodiment, a condition for determining, as themain object, a foreground present within a predetermined angular rangecentered on the virtual camera viewpoint is set.

Under conditions 804 to 806, evaluation conditions are set which arecombinations of change in the position of the virtual camera, theorientation of the virtual camera, and the direction of apparent motionof the background under the viewing condition. Note that the directionof the amount of background motion takes the viewing condition obtainedin S603 into consideration since there can be a change in direction(such as viewing the video sideways) depending on the viewing condition.Under a condition 801, conditions on the amount of apparent motion ofthe main object in the virtual viewpoint video are stored. By combiningthe above conditions, an evaluation condition corresponding to the videoscene is determined.

Here, the evaluation conditions in FIG. 8 cover evaluation conditionsfor the video scenes of the virtual viewpoint videos illustrated inFIGS. 7B to 7D. Specifically, the evaluation conditions in FIG. 8 coverevaluation conditions for scenes in virtual viewpoint videos in which,in the user's view, the change in position of the main object on thescreen is small and the change of the background on the screen is large(the background moves). The condition 801 covers conditions on theamount of change of the main object area (foreground) (conditions eachdefining whether the amount of change of the foreground is smaller thana predetermined threshold value). A condition 802 cover conditions onthe amount of change of the background (conditions each defining whetherthe amount of change of the background is smaller than a predeterminedthreshold value). In this case, the visually-induced motion sicknessevaluation is performed on the basis of the amounts of apparent motionof the background listed under the condition 802. Specifically, in thecase of using an evaluation condition listed in FIG. 8, the video sceneis determined by the condition 801 and the conditions 804 to 806, and anevaluation condition corresponding to that video scene is determined byreferring to the condition 802.

In the example presented above, the video scene is determined on thebasis of the amount of motion of the foreground and the combination ofthe change in position of the virtual camera and the orientation of thevirtual camera, and the amount of motion of the background correspondingto that video scene is used as the evaluation condition. Note, however,that the present invention is not limited to this. The video scene maybe determined on the basis of the amount of motion of the background andthe combination of the change in position of the virtual camera and thedirection of the virtual camera, and the amount of motion of theforeground corresponding to that video scene may be used as theevaluation condition. Alternatively, the video scene may be determinedon the basis of the amounts of motion of the foreground and thebackground and the combination of the change in position of the virtualcamera and the direction of the virtual camera, and the amounts ofmotion of the foreground and the background corresponding to that videoscene may be used as the evaluation condition.

Also, the condition for distinguishing between scenes with differentdegrees of visually-induced motion sickness, and the evaluationcondition for evaluating visually-induced motion sickness may be acondition for any of changes in speed, acceleration, deceleration,direction, and position of the foreground or the background, and acondition for any of changes in speed, acceleration, deceleration,direction, position, and orientation of the virtual camera.

Also, in the example described, the video display size in which thevirtual viewpoint video is to be viewed by the user, and the distancebetween the user and the displayed video are referred to as the viewingcondition. However, other viewing conditions may be taken intoconsideration. These other viewing conditions may include the displaydevice's type, brightness, and dynamic range, and the conditions ofsurroundings in the viewing environment (such as lighting). The viewingcondition may also include profile information on user characteristicssuch as the age and sex of the user who is to view the virtual viewpointvideo and the user's susceptibility to motion sickness, and so on.Different visually-induced motion sickness evaluation conditionscorresponding to these viewing condition elements may be stored in thevideo evaluation condition storage unit 335, and an appropriateevaluation condition may be selected in accordance with the viewingcondition.

Referring back to FIG. 6, the description will be continued. In S607, onthe basis of the visually-induced motion sickness evaluation conditionobtained in S606, the video evaluation unit 333 evaluates whether thecondition is satisfied (whether the video causes visually-induced motionsickness). With the video scene examples in FIG. 7B to 7D, which one ofFIGS. 7B to 7D the video scene matches is determined, and an evaluationcondition corresponding to that video scene is obtained in S606.Specifically, a threshold value for the amount of apparent motion of thebackground in the video is obtained as the evaluation condition. InS607, the video evaluation unit 333 determines the amount of motion ofthe background on the basis of the virtual camera parameters and, if theamount of apparent motion of the background in the virtual viewpointvideo corresponding to those virtual camera parameters is more than thethreshold value obtained in S606, the video evaluation unit 333determines that the evaluation target frame is a frame image that causesvisually-induced motion sickness. The result of the determinationcontains the frame number of the frame image that exceeded theevaluation condition (i.e. the frame image determined to causevisually-induced motion sickness). The result of the determination alsocontains the foreground image area or background image area thatexceeded the evaluation condition (i.e. the foreground image area orbackground image area determined to cause visually-induced motionsickness).

<Visually-Induced Motion Sickness Warning Display>

FIG. 9 illustrates an example of a UI screen 900 displaying the resultof a visually-induced motion sickness evaluation on a virtual viewpointvideo. The UI screen 900 is displayed on the display unit 104.

In the UI screen 900 for the result of a visually-induced motionsickness evaluation, a timeline 905 represents a frame rangecorresponding to the playback period of the virtual viewpoint image. Apointer 908 designates a frame image displayed in a frame image displayarea 901. Also, any frame for which a visually-induced motion sicknessunder the set viewing condition should be warned of is displayed in arecognizable manner on the timeline 905, like frame warning displays 906and 907. A visually-induced motion sickness evaluation value displayarea 904 indicates a visually-induced motion sickness evaluation elementcalculated for each frame. In the visually-induced motion sicknessevaluation value display area 904, for example, the value of apparentspeed in the video, the amount of change in motion vector in the video,or the like is displayed. The likelihood of motion sickness in thevisually-induced motion sickness evaluation result may be quantified andthat quantified result may be displayed in the visually-induced motionsickness evaluation value display area 904.

A warning image display area 902 is synchronized with the frame imagedisplayed in the frame image display area 901. In the warning imagedisplay area 902, an image area 903 that can be a cause ofvisually-induced motion sickness under the set viewing condition in theframe image displayed in the frame image display area 901 ishighlighted. Video playback operation buttons 909 to 913 are buttons forperforming playback operations such as starting and stopping playback ofthe virtual viewpoint video displayed in the frame image display area901.

Note that the description has been given by taking as an example a casewhere a single viewing condition is set for the sake of description.However, a plurality of viewing conditions may be set, and avisually-induced motion sickness evaluation may be performed for each ofthem. Moreover, in the UI screen 900, sets of the warning image displayarea 902, the visually-induced motion sickness evaluation value displayarea 904, and the timeline 905 for the respective viewing conditions maybe displayed side by side for the respective conditions. Alternatively,a button to switch between the viewing conditions may be provided, andthe UI screen 900 may be switched in response to an operation on thebutton by the user.

In the example presented, the visually-induced motion sicknessevaluation in this embodiment involves determining whether a videocauses visually-induced motion sickness, on the basis of whether itexceeds a visually-induced motion sickness evaluation condition.However, it is possible to employ a configuration that outputs anevaluation value among graduated numerical values obtained by convertingdegrees of visually-induced motion sickness caused by types of videoscene and amounts of motion of the foreground or background. Forexample, since visually-induced motion sickness occurs in a case wherethe amount of motion exceeds a threshold value, it is possible to outputan evaluation value obtained by quantifying intervals up to the point atwhich the amount of motion exceeds the threshold value. In this example,the higher the evaluation value, the more likely visually-induced motionsickness occurs. In a case where the amount of motion further increasesafter exceeding the threshold value, motion sickness conversely tendsnot to occur. Thus, after the amount of motion exceeds the thresholdvalue, the evaluation value may be caused to decrease the further theamount of motion increases. Also, the tendency of occurrence ofvisually-induced motion sickness varies by the type of motion (verticalmotion, horizontal motion, rotational motion, etc.). Thus, an evaluationvalue corresponding to that tendency may be output.

As described above, in the process performed in this embodiment, theamounts of motion of the foreground image area and the background imagearea in an evaluation target frame in a virtual viewpoint video areobtained. Also, the virtual camera path in the evaluation target frameis obtained. Then, an evaluation condition corresponding to these piecesof information is determined. For example, the visually-induced motionsickness evaluation unit 330 determines the video scene of theevaluation target frame and determines an evaluation condition suitablefor video scene. Then, the visually-induced motion sickness evaluationunit 330 evaluates visually-induced motion sickness due to theevaluation target frame by using the determined evaluation condition. Byperforming such a process, an accurate visually-induced motion sicknessevaluation suitable for the video scene is performed. In virtualviewpoint videos, there can be video scenes that are similar inforeground and background motions but are totally different in virtualcamera path. According to this embodiment, which one of the scenes ofFIGS. 7B to 7D the video scene is, for example, is determined by usingthe foreground or background motion and the virtual camera path. Then,by using an evaluation condition suitable for that video scene,visually-induced motion sickness is evaluated accurately.

Embodiment 2

In embodiment 1, description has been given of an image processingapparatus that evaluates visually-induced motion sickness due to aninput virtual viewpoint video and displays a warning for any frame rangeand image area in the virtual viewpoint video where the degree ofvisually-induced motion sickness exceeds a predetermined thresholdvalue. In this embodiment, description will be given of a configurationenabling the user to easily correct a virtual camera parameter(s) toreduce visually-induced motion sickness.

<Configuration of Image Processing Apparatus>

FIG. 10 is a diagram illustrating an example of the logicalconfiguration of an image processing apparatus 100 in embodiment 2. Ithas the configuration described in embodiment 1 with a virtual cameraparameter correction unit 1000 further included therein. The otherfeatures of the configuration are similar to those in embodiment 1. Thevirtual camera parameter correction unit 1000 determines a recommendablevirtual camera parameter(s) (within a range that reducesvisually-induced motion sickness) by using the result of the evaluationperformed by the visually-induced motion sickness evaluation unit 330.For example, the virtual camera parameter correction unit 1000 changes avirtual camera parameter(s) in a stepwise manner. The visually-inducedmotion sickness evaluation unit 330 evaluates visually-induced motionsickness due to a virtual viewpoint video generated using the changedvirtual camera parameter. By using the result of that evaluation, thevirtual camera parameter correction unit 1000 determines a virtualcamera parameter within a range in which visually-induced motionsickness does not occur. The user can set the virtual camera parameterwithin that determined range. The virtual camera parameter correctionunit 1000 corrects the virtual camera parameter by replacing (changing)the virtual camera parameter stored in the virtual camera parameterstorage unit 310 with the set virtual camera parameter. Then, a virtualviewpoint video using the corrected virtual camera parameter isgenerated. Meanwhile, instead of replacing the stored virtual cameraparameter, another virtual camera parameter representing a virtualviewpoint determined on the basis of a user operation may beadditionally stored. The configuration of the virtual camera parametercorrection unit 1000 will be described below.

The virtual camera parameter correction unit 1000 includes a correctionrange determination unit 1001, a correction range storage unit 1002, anda correction parameter setting unit 1003. On the basis of the virtualcamera parameter used for a virtual viewpoint video determined to causevisually-induced motion sickness, the correction range determinationunit 1001 determines a possible virtual camera parameter correctionrange in which visually-induced motion sickness does not occur. Thedetermined possible correction range is stored in the correction rangestorage unit 1002. The possible virtual camera parameter correctionrange is presented to the user through the display unit 104. Thecorrection parameter setting unit 1003 sets a corrected value of thevirtual camera parameter designated within the possible correction rangeby the user. Specifically, the correction parameter setting unit 1003stores that corrected value (corrected virtual camera parameter) in thevirtual camera parameter storage unit 310. The virtual viewpoint videogeneration unit 300 generates a virtual viewpoint video with reducedvisually-induced motion sickness by using the corrected virtual cameraparameter. In this embodiment, the correction of the virtual cameraparameter will be described by taking as an example a process ofcorrecting the coordinates of the virtual camera.

FIG. 11 is a flowchart describing a series of processes performed in theimage processing apparatus 100 after performing a visually-inducedmotion sickness evaluation on a virtual viewpoint video to correct thevirtual camera parameter in a frame range determined to causevisually-induced motion sickness.

In S1101, the virtual camera parameter correction unit 1000 obtains thevirtual camera parameter used for the virtual viewpoint video. Thevirtual camera parameter includes the camera coordinate values, thecamera orientation, the focal length, and so on in each of the videoframes (frame images) along the time axis inside the virtual space forgenerating the virtual viewpoint video.

FIGS. 12A and 12B are an example of a UI illustrating the coordinates ofthe virtual camera included in the obtained virtual camera parameter(hereinafter, referred to as the virtual camera coordinates). In thisembodiment, a three-dimensional virtual space is used to generate thevirtual viewpoint video. On a UI screen 1200, plane coordinates in thevirtual space are displayed as a XY plane. On a UI screen 1201, a YZplane in the virtual space is displayed with the height in the virtualspace being the Z axis. The virtual camera path in the virtual viewpointvideo is illustrated by a start point 1204, an end point 1205, and atrack 1206 representing the change in virtual camera coordinates betweenthem.

In S1102, the virtual camera parameter correction unit 1000 obtains theframe range evaluated to cause visually-induced motion sickness from theresult of the visually-induced motion sickness evaluation on the virtualviewpoint video data.

In S1103, the correction range determination unit 1001 changes thevirtual camera coordinates in the first frame image to be processed inthe frame range evaluated to cause visually-induced motion sickness, bya given coordinate data interval. Note that in this embodiment, a limithas been set in advance on the range within which to change the virtualcamera coordinates, and the given coordinate data interval is set to avalue within the limit. In this embodiment, as also described later, thevirtual camera coordinates are changed from the first coordinatesobtained in S1101 to second coordinates, which are differentcoordinates, and visually-induced motion sickness due to a virtualviewpoint video generated using the changed second coordinates isevaluated. Then, if the virtual viewpoint video does not causevisually-induced motion sickness, the second coordinates are determinedas a correction range. Thereafter, the virtual camera coordinates arechanged to third coordinates, which are different from the secondcoordinates, and similar processes are repeated. As a result, thecorrection range for the first frame image is determined. Theseprocesses are performed for all frame images in the frame range.

FIGS. 13A and 13B are diagrams explaining an example of setting a limiton the range within which to change the virtual camera coordinates. FIG.13A illustrates an XY plane coordinate system 1300 in the virtual spaceof the virtual viewpoint video. In the XY plane coordinate system 1300,virtual camera coordinates 1304 on an input virtual camera path 1306 aredisplayed. FIG. 13B is a frame image 1301 in the virtual viewpoint videocorresponding to the virtual camera coordinates 1304. A virtual cameraviewpoint area 1302 in the frame image 1301 represents the same area asa virtual camera viewpoint area 1302 in the XY plane coordinate system1300. The virtual camera viewpoint area 1302 is an area covering apredetermined range including the gazing point. Assume that the virtualcamera coordinates 1304 are to be changed to different coordinates tocorrect the virtual camera parameter. In this case, a predeterminedlimit is set on the amount of movement from the virtual camera viewpointarea 1302 to a virtual camera viewpoint area 1303, which is thedestination. In other words, the virtual camera viewpoint area 1303,which is the destination, is set within the range defined by thepredetermined limit. In this way, a limit can be set on the change ofthe virtual camera coordinates within such a range the frame image 1301from the virtual camera viewpoint does not change greatly.

In S1104, the correction range determination unit 1001 generates avirtual viewpoint video using the corrected virtual camera coordinates.The correction range determination unit 1001 can generate the virtualviewpoint video by a process similar to the process performed in thevirtual viewpoint video generation unit 300. Alternatively, thecorrection range determination unit 1001 may cause the virtual viewpointvideo generation unit 300 to generate the virtual viewpoint video.

In S1105, the correction range determination unit 1001 performs avisually-induced motion sickness evaluation on the virtual viewpointvideo generated using the changed virtual camera coordinates. Thecorrection range determination unit 1001 can perform thevisually-induced motion sickness evaluation by a process similar to theprocess performed in the visually-induced motion sickness evaluationunit 330. Alternatively, the correction range determination unit 1001may cause the visually-induced motion sickness evaluation unit 330 toperform the visually-induced motion sickness evaluation.

In S1106, the correction range determination unit 1001 determineswhether the result of the visually-induced motion sickness evaluation inS1105 exceeds a predetermined threshold value. If the result of theevaluation exceeds the predetermined threshold value, the correctionrange determination unit 1001 returns to the process in S1103, in whichit further changes the virtual camera coordinates by the given interval,and repeats the processes in S1104 and S1105.

If the result of the evaluation does not exceed the predeterminedthreshold value, then in S1107, the correction range determination unit1001 stores the virtual camera coordinates changed in S1103 in thecorrection range storage unit 1002 as virtual camera coordinates that donot cause visually-induced motion sickness.

In S1108, the correction range determination unit 1001 determines thevirtual camera coordinates changed in S1103 are within the predeterminedlimit, and repeats the processes in and after S1103 are repeated if thevirtual camera coordinates are within the limit. If the virtual cameracoordinates are outside the limit, then in S1109, the correction rangedetermination unit 1001 determines whether the processes in S1103 toS1108 have been executed on all frames in the frame range evaluated tocause visually-induced motion sickness.

If the processes have not been completed for all frames, the correctionrange determination unit 1001 updates the target frame in S1113 andrepeats the processes in and after S1103. If the processes have beencompleted for all frames, then in S1110, the virtual camera parametercorrection unit 1000 displays the possible virtual camera coordinatecorrection range stored in the correction range storage unit 1002 to theuser.

In S1111, the correction parameter setting unit 1003 sets a virtualcamera parameter in the frame range evaluated to cause visually-inducedmotion sickness, within the possible correction range on the basis of adesignation from the user. Note that although the virtual cameraparameter in the frame range evaluated to cause visually-induced motionsickness is determined on the basis of a designation from the user, thismay be omitted. Specifically, the image processing apparatus 100 maychange the virtual camera parameter in the frame range evaluated tocause visually-induced motion sickness to a virtual camera parameterthat can be evaluated not to cause visually-induced motion sicknessautomatically without receiving a designation from the user. As aresult, the virtual camera parameter is corrected. In S1112, thecorrection parameter setting unit 1003 stores the corrected virtualcamera parameter in the virtual camera parameter storage unit 310. Bythe above step, the process in this flowchart ends.

FIGS. 14A and 14B are diagrams illustrating examples of UI screens eachdisplaying a possible correction range. A UI screen 1400 illustrated inFIG. 14A is a UI screen displaying a possible correction range for thevirtual camera coordinates illustrated in FIG. 12A. On the UI screen1400, the range defined between the dotted lines is displayed as apossible correction range 1407 for the track 1206 of the virtual cameracoordinates on a YZ plane in the virtual space.

The user can correct the virtual camera coordinates by changing thetrack 1206 on the UI screen 1400 such that the track 1206 is locatedwithin the possible correction range 1407. The corrected virtual cameracoordinates are stored in the virtual camera parameter storage unit 310.

Note that in this embodiment, description has been given of the processin which the virtual camera position is changed for virtual cameraparameter correction. However, a similar process can be used as avirtual camera parameter correction process for correcting a differentvirtual camera parameter.

FIG. 14B illustrates a UI screen 1401 for correcting the playback speedof a virtual viewpoint video as an example of the correction of adifferent virtual camera parameter. The UI screen 1401, illustrated inFIG. 14B, illustrates the virtual camera coordinates illustrated in FIG.12B. The UI screen 1401 illustrates a possible correction range for theplayback speed. On the UI screen 1401, each white circle 1408 on thetrack 1206 of the virtual camera coordinates in the virtual viewpointvideo indicates the camera coordinates in a frame. The maximum playbackspeed of the virtual viewpoint video is the frame rate of the inputvideo data. Here, during playback, the same frame is assignedsuccessively to a predetermined frame(s) in the virtual viewpoint video.In this way, the playback speed can be lowered. Thus, by setting thenumber of same frames to be reproduced at desired virtual cameracoordinates in a virtual viewpoint video, it is possible to control theplayback speed of the virtual viewpoint video while changing the virtualcamera coordinates. On the UI screen 1401, the number of same frames atthe camera coordinates in each frame is indicated by the size of a whitecircle 1409. Thus, on the UI screen 1401, a possible playback speedcorrection range in which visually-induced motion sickness does notoccur (upper limit frame rate) is displayed by varying the size of thewhite circle 1409 on the basis of the result of the visually-inducedmotion sickness evaluation.

Note that in the processes in S1106 to S1110 in the flowchart of FIG.11, a plurality of threshold values can be set for a visually-inducedmotion sickness evaluation result, and possible correction rangescorresponding to the plurality of threshold values may be calculated.Then, the possible correction ranges corresponding to the plurality ofthreshold values can be displayed on the UI screens 1400 and 1401.

Note that the virtual camera parameter correction unit 1000 may beconfigured to obtain profile information on user characteristics andperform a correction process suitable for the user's profile in a casewhere the user corrects any virtual camera parameter. For example, in acase where the user is a professional in video production and editing,the number of virtual camera parameters to be corrected and the possiblecorrection ranges may be set to wide ranges covering ranges in whichmild visually-induced motion sickness occurs, for example. In this way,the degree of freedom in correction can be enhanced. In short, in thelimiting of a possible correction range, that limit may be set to belooser than a first limit (the possible correction range may bewidened).

On the other hand, for a normal user without a video productionexpertise or knowledge on visually-induced motion sickness, the numberof virtual camera parameters to be corrected and the possible correctionranges may be limited to ranges in which the likelihood of occurrence ofvisually-induced motion sickness is low. In this way, a virtualviewpoint video with reduced visually-induced motion sickness can begenerated. In short, in the limiting of a possible correction range,that limit may be set to be stricter than the first limit (the possiblecorrection range may be narrowed).

As described above, according to this embodiment, by using the result ofa visually-induced motion sickness evaluation on a virtual viewpointvideo, it is possible to present to the user a possible correction rangerepresenting a range in which visually-induced motion sickness does notoccur. Then, the user can correct the virtual camera parameter withinthe presented possible correction range. With such a process, the usercan correct the virtual camera parameter within a range in whichvisually-induced motion sickness does not occur. Thus, the user canavoid an operation of re-setting the virtual camera parameter of thevirtual viewpoint video.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

According to the present disclosure, visually-induced motion sicknessdue to a virtual viewpoint video can be accurately evaluated.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-036651, filed Mar. 1, 2018, which is hereby incorporated byreference wherein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising: afirst obtaining unit configured to obtain viewpoint informationindicating change of a virtual viewpoint corresponding to a virtualviewpoint video generated on a basis of a plurality of images capturedfrom a plurality of directions with a plurality of image capturingapparatuses; a second obtaining unit configured to obtain conditioninformation indicating a condition associated with an amount of changeof a background in the virtual viewpoint video; a determination unitconfigured to determine whether the viewpoint information obtained bythe first obtaining unit satisfies the condition indicated by thecondition information obtained by the second obtaining unit; and anoutput unit configured to output information according to a result ofthe determination by the determination unit.
 2. The informationprocessing apparatus according to claim 1, wherein the informationoutput from the output unit is information indicating an evaluation onvisually-induced motion sickness due to the virtual viewpoint video. 3.The information processing apparatus according to claim 2, wherein on abasis of the viewpoint information obtained by the first obtaining unit,the determination unit determines whether the amount of change of thebackground in the virtual viewpoint video corresponding to the viewpointinformation exceeds a threshold value corresponding to the conditioninformation obtained by the second obtaining unit, and the output unitoutputs information indicating that the virtual viewpoint video causesvisually-induced motion sickness, in a case where the determination unitdetermines that the amount of change of the background exceeds thethreshold value.
 4. The information processing apparatus according toclaim 1, wherein the condition information obtained by the secondobtaining unit further indicates a condition associated with an amountof change of a foreground in the virtual viewpoint video.
 5. Theinformation processing apparatus according to claim 1, wherein thecondition information obtained by the second obtaining unit indicates acondition corresponding to an environment in which the virtual viewpointvideo is to be viewed by a user.
 6. The information processing apparatusaccording to claim 1, wherein the condition information obtained by thesecond obtaining unit indicates a condition corresponding to a patternof movement of the virtual viewpoint.
 7. The information processingapparatus according to claim 1, further comprising a display controlunit configured to cause a display unit to display an imagecorresponding to the information output from the output unit.
 8. Theinformation processing apparatus according to claim 7, wherein thedisplay control unit causes the display unit to display informationrepresenting a virtual viewpoint recommended on a basis of theinformation output from the output unit.
 9. The information processingapparatus according to claim 1, wherein the output unit outputsinformation indicating the result of the determination by thedetermination unit and information for identifying a frame in thevirtual viewpoint video according to the result of the determination, asthe information according to the result of the determination.
 10. Theinformation processing apparatus according to claim 1, wherein theoutput unit outputs information indicating the result of thedetermination by the determination unit and information indicating anarea in the virtual viewpoint video in which the amount of change of thebackground is more than or equal to a threshold value, as theinformation according to the result of the determination.
 11. Theinformation processing apparatus according to claim 1, wherein thedetermination unit determines whether the viewpoint informationsatisfies the condition indicated by the condition information, for eachof a plurality of periods in a playback period of the virtual viewpointvideo, and the output unit outputs information according to a result ofthe determination on each of the plurality of periods in the playbackperiod.
 12. The information processing apparatus according to claim 1,further comprising a changing unit configured to change the viewpointinformation obtained by the first obtaining unit, on a basis of theinformation output from the output unit.
 13. The information processingapparatus according to claim 1, further comprising a determination unitconfigured to determine a playback speed of the virtual viewpoint videoon a basis of the information output from the output unit.
 14. Theinformation processing apparatus according to claim 1, furthercomprising: a third obtaining unit configured to obtain other viewpointinformation indicating a virtual viewpoint determined on a basis of auser operation, after the output unit outputs the information accordingto the result of the determination; and a generation unit configured togenerate a virtual viewpoint video on a basis of the other viewpointinformation determined by the third obtaining unit.
 15. The informationprocessing apparatus according to claim 14, wherein the third obtainingunit determines the other virtual viewpoint on a basis of an inputaccording to the user operation and user information.
 16. Theinformation processing apparatus according to claim 1, wherein thebackground is an area in the virtual viewpoint video other than aforeground corresponding to a predetermined object, and the backgroundin the virtual viewpoint video changes in accordance with change of thevirtual viewpoint corresponding to the virtual viewpoint video.
 17. Theinformation processing apparatus according to claim 1, wherein theviewpoint information obtained by the first obtaining unit indicates aposition and direction of the virtual viewpoint at each of a pluralityof time points.
 18. The information processing apparatus according toclaim 1, wherein the determination unit determines whether the viewpointinformation satisfies the condition indicated by the conditioninformation, on a basis of a position and direction of the virtualviewpoint indicated by the viewpoint information in a three-dimensionalspace and a position of an object in the three-dimensional space.
 19. Aninformation processing method comprising: obtaining viewpointinformation indicating change of a virtual viewpoint corresponding to avirtual viewpoint video generated on a basis of a plurality of imagescaptured from a plurality of directions with a plurality of imagecapturing apparatuses; obtaining condition information indicating acondition associated with an amount of change of a background in thevirtual viewpoint video; determining whether the obtained viewpointinformation satisfies the condition indicated by the conditioninformation; and outputting information according to a result of thedetermination.
 20. A non-transitory computer readable storage mediumstoring a program which performs an image processing method, the methodcomprising: obtaining viewpoint information indicating change of avirtual viewpoint corresponding to a virtual viewpoint video generatedon a basis of a plurality of images captured from a plurality ofdirections with a plurality of image capturing apparatuses; obtainingcondition information indicating a condition associated with an amountof change of a background in the virtual viewpoint video; determiningwhether the obtained viewpoint information satisfies the conditionindicated by the condition information; and outputting informationaccording to a result of the determination.